Eye-tracking and Protocol Analysis of Expertise in Air Traffic Control Situation Awareness

Analyses of eye tracking and think-aloud protocol data were performed to examine novice-expert differences in perceptual and cognitive aspects of air traffic controllers’ situation awareness. In Experiment 1, participants were asked to perceive situations that were manipulated by situation complexity. In Experiment 2, protocol analysis was performed to extract different task models and strategy models as a function of expertise. Then delayed-recall task and interviews about air control plans for the recalled situations were also executed. Results showed that expert controllers concentrate only on several critical features and have their own strategies to reduce mental workloads. 2000 3000 4000 5000 6000 7000 Simple Complex Fi xa tio ns Novices Intermediates Experts Introduction This research focused on novice-expertise differences in visual attention and think-aloud protocols of air traffic situation awareness. Situation awareness should be central to successful air traffic control (ATC) performance (Endsley, 1995). The objective of this research was to advance our understanding of controller expertise and cognitive process in ATC situations varying with the demand of workload. In this study, eye-tracking and protocol analyses were executed since the eye movements relating to think-aloud protocols should serve as effective clues to understand cognitive process of controller performance (Ware & Mikaelian, 1987; Ericsson & Charness, 1994). Experiment 1 To examine controllers' perceptual information processing pattern as a function of expertise and complexity of traffic flow, we employed a 3 x 2 mixed factorial design with a between-participants variable of controller expertise (expert, intermediate, novice) and a within-participants variables of complexity of traffic flow (simple, complex). The ATC situations were represented with radar screen pictures, which consisted of four situations varying with complexity of traffic flow. 0 50 100 150 200 250 Simple Complex D ur at io n (m se c) Novices Intermediates Experts Methods Seventeen field controllers from Seoul Approach Control Center participated in this experiment. The participants fell into three groups, each with different levels of ATC experience: 4 experts, 8 intermediates, and 5 novices. Experiment procedure was divided into two stages; preparative stage and executive stage. In preparative stage, participants put on an eyetracker designed to record participants' eye movements. In executive stage, participants were to view four radar screen pictures each for 30 seconds. The procedure was repeated four times per participant in accordance with four screen pictures. Results and Discussion The Number of Fixations. As shown in Figure 1, there was a significant interaction between expertise and complexity of traffic flow, F(2,14) = 3.773, p < .05, η = .350. For experts, the number of fixations was smaller for the complex traffic flow than for the simple traffic flow. For novices, in contrast, the number of fixations was greater for the complex traffic flow than for the simple traffic flow. Figure 1: Average number of fixations. The Duration of Each Fixation. A significant interaction between expertise and complexity of traffic flow was found, F(2.14) = 6.328, p < .05, η = .475. As shown in Figure 2, the expert group spent more time gazing at targets in the complex traffic flow than that in the simple traffic flow whereas the novice group spent less time gazing at targets in the complex traffic flow than that in the simple traffic flow. Figure 2: Average duration of each fixation. The Distance of Saccades between Fixations. There was a significant effect of expertise on the distance of saccades, F(2,14) = 3.452, p = .06, η = .330. And a significant interaction between expertise and complexity of traffic flow was confirmed, F(2,14) = 7.148, p < .05, η = .505. The expert group showed an increased distance of saccades in the complex traffic flow, which was not the case for the other groups, as shown in Figure 3.